American Journal of Business Education - Fourth Quarter 2008 


Volume 1, Number 2 


Digital Assignments: 

The Importance Of Pedagogical Context 

S. Christoffersen, Philadelphia University, USA 
E. Granitz, Long Island University, USA 


ABSTRACT 

We assess students’ appreciation and use of various tools designed to extend effective learning 
beyond the classroom. For a given tool, results vary depending upon the overall pedagogical 
approach of the course. 


INTRODUCTION 


otivating student interest is one of the most difficult challenges for educators today, especially 
when it entails time outside of the classroom. This paper suggests that high technology 
components can enhance student engagement but that the effectiveness of a given high technology 
component is dependent on the overall learning environment, specifically the degree to which technology is 
integrated throughout the course. 



We examine two Principles of Microeconomics course models. One is a traditional course with a textbook, 
midterm exams and a final exam; the students meet in the classroom on a regular schedule, read a paper textbook 
and use pencils for problem solving. In the high tech model, the students still meet for the regular hours each week 
but the textbook and study tools are all online (students are encouraged to purchase an unembellished hard copy of 
the online textbook). 


We introduce the same web based exercises to both course types. We then assess the usefulness of the 
given exercises and the impact of the overall course structure. In addition, we examine the students’ responses to 
other aspects of the course, given the different approaches. 

TWO LEARNING ENVIRONMENTS 


In general, a student is exposed to material by three main conduits: the student is asked to listen to lectures, 
read subject matter, and solve assigned problems. In both courses in this study, the students are expected to read a 
textbook and complete assigned homework. In the traditional course, the students have a study guide to stimulate 
learning outside of the classroom. In the high tech course the students are required to complete an open book online 
quiz prior to class. The purpose of the quiz is to force the students to read the online book and think about the 
material before they attend the lecture. 

In the high tech course, the study guide materials are all online including flash cards, matching and some 
graphing exercises. There are also numerical supply and demand problems online which have answers available for 
practice, and mimic the online quiz, the results of which are reported to the instructor. In addition to these relatively 
standard ancillaries, the instructor has created a number of online Flash based problems such as graphical supply 
and demand problems, graphical industry models, and numerical and graphical trade problems. Only the publisher’s 
quizzes are specifically assigned as homework to be graded as the correct completion rates of the newly created 
problems can not be tracked. The students are strongly recommended to use the instructor’s practice problems as 
they mimic the types of problems and graphs they must answer on their in class exams. The technology aspects of 
the course extend into the classroom where the lecture/discussion is supported by the use of a tablet PC with a 
projector and an internet connection. 
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Both instructors use a lecture / discussion format in the classroom. Both classes have the students do some 
group problem solving in class as well. Such collaborative problem-solving in class addresses diverse learning 
styles and allows earlier assessment of the students’ strengths and weaknesses. The high tech course is designed to 
encourage active learning outside the classroom through the discipline imposed by the online quizzes. The 
traditional class regularly incorporates active learning exercises throughout the semester, fostering more experiential 
learning opportunities in the classroom. Both efforts are geared to the same goal, engaging students in the learning 
process. 

LITERATURE ON PEDAGOGY 

The effectiveness of active learning is supported by a large body of literature (Cross, 1987). Economics, in 
particular, “requires active participation by students” according to Siegfried, et al., (1991). Hansen, Salemi, and 
Siegfried (2001) stress the need to change the methods that we use to teach the principles of economics classes; 
despite continuing pressure to improve pedagogy, Becker and Watts (1996) find that active learning is not used in 
the majority of introductory courses. This may be because the costs to the instructor of changing teaching 
techniques may outweigh the benefits. “There is a considerable cost in changing the manner in which we teach and 
the incentives to change how we teach do not exist in most institutions.” (Buckles, 2001, p.2) Furthermore, costs of 
innovative pedagogy can include heightened career risk. In surveys, students’ responses to innovative pedagogy 
include dubious comments such as they are not “used to it” and prefer fewer resources. While this may not be a 
majority opinion nor an accurate assessment of the value of the pedagogy, negative student comments can be costly 
to an instructor. 

One indicator of the potential cost to a faculty member for implementing innovative pedagogy is the initial 
student response found in the high tech course: a high drop rate at the onset of the course. Some students will be 
frustrated with the set up costs of using the technology. Some students will decide that taking weekly quizzes, in 
addition to their other work, is too demanding. Furthermore, to take the online quizzes, the students have to read the 
text and/or look for the answers and try and solve the problems posed. The expectation is that students will grasp the 
definitions and then contemplate the problems for each chapter. These activities required active thoughtful learning 
on the part of the students. The problem arises because the quizzes are immediately graded and diligent students, 
just working from the text with no additional explanations, often earn grades between 50% and 70%. As explained 
to the students, grades in this range or above would be good scores. Not all students clearly understand how the 
quizzes are curved and so some drop the course early in the semester thinking they did not adequately understand 
the material. This higher drop rate may be an issue to probationary faculty in particular. 

Under the umbrella of active learning is collaborative learning: “the instructional use of small groups” 
(Johnson et al. (1991). Collaborative learning is particularly important for the more diverse class. Lage and Treglia 

(1998, p.42) state that “Cooperative learning,....and other forms of nonhierarchical classroom activities . offer 

possibilities for enhancing the persistence of women in economics as well as leading to improved performance for 
students with diverse learning styles.” The diversity of learning styles is described in research showing “60% of 
American students are visual dominant learners, 37% are auditory dominant learners, and 3% are kinesthetic 
dominant learners” (Rogers, 2000, p.20). The traditional course relies more on collaborative learning and activities 
in the classroom while the high tech course relies on the visual online experience outside of the classroom and 
enhanced discussion in the class. 

Literature shows that active and collaborative learning strategies enhance retention and understanding of 
complex concepts. The challenges facing today’s students require them to acquire skills more profound than simple 
memorization. “Now the real world demands individuals use high order reasoning skills to solve complex 

problems.Learners must now be viewed as proactive participants in learning, actively seeking ways to analyze, 

question, interpret, and understand their ever-changing environment” (Newby et. al., p.6, emphasis added). 

Collaborative learning is more effective when done regularly. In courses where the students don’t routinely 
work together, they militate when occasionally requested to form groups and work together. In the “traditional” 
classroom described in this paper, the students form groups to conduct exercises or solve problems regularly. 
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beginning with the first class meeting. As the semester progresses, the students readily work together and do not 
need urging to get results; this would not be the case in a classroom where students generally sit passively and listen 
to the lecture. In the same way, the high tech exercises are better received by the students in the more consistently 
high tech learning environment, as shown by the survey results that follow. Learning is enhanced by immediate 
feedback; the traditional classroom has collaborative feedback in real time, the high tech has immediate online 
feedback. Despite their differences, both pedagogies embrace student centered learning and improved motivation. 
Both are effective in sundry ways. 

ASSESSMENT 

Learning theorists have long told us the extreme importance of motivation.” (Buckles, 1998, p.227, 
emphasis added). Going as far back as ancient Rome, the poet, Horace, wrote “Instruct by pleasing”. That active 
learning leads to a positive perception is described by Light (1990), “Active engagement with other learners is a 
characteristic of educational experiences that are highly personally fulfilling.” Vail (1994) discusses the importance 
of a positive attitude in her book. Emotion: the On/Off Switch for Learning. While learning outcomes remain 
difficult to assess, students’ attitudes are readily surveyed. We use such surveys to initially indicate the 
effectiveness of the pedagogies we introduced to the different class types. 

Survey data of the classes show the students’ levels of receptiveness to various aspects of both courses. 
The surveys were distributed in the last week of classes, 54 students returned the survey in the traditional course and 
33 in the high tech course. The survey questions are presented below with the tabulated responses. 

Assessment Of Teaching Tools Survey Results: 

The first major difference between the two courses is the percent of students who read the textbook whether 
online or the hard copy. Overall we see that 85% of the high tech students read the book compared to 62% in the 
traditional course. 


Table 1 



Traditional 

High Tech ! 


Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Did you read the textbook? 

33 

20 

62% 

38% 

28 

5 

85% 

15% 

Do you think the textbook helped you to participate 
more in class? 

12 

39 

24% 

76% 

14 

18 

44% 

56% 

Do you think the textbook helped you perform better on 
tests? 

21 

28 

43% 

57% 

8 

23 

26% 

74% 


As the table below shows, 57% of the high tech students read all or most of the assigned chapters compared 
to only 30% in the traditional class. In both courses, about a quarter of the students didn’t try to read the book or got 
discouraged. 


Table 2 


I read the book (circle one answer) 

Traditional 

High Tech I 

a. All of the assigned chapters 

2 

4% 

10 

30% 

b. most of the assigned chapters 

14 

26% 

9 

27% 

c. skimmed the assigned chapters 

23 

43% 

6 

18% 

d. tried to read but got discouraged 

12 

22% 

5 

15% 

e. did not buy the book 

3 

6% 

3 

9% 


54 


33 
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Examining the students’ motivation for reading the text we can see that the high tech students were largely 
motivated by the required chapter quizzes. The traditional students are also motivated by the “quizzes” but for this 
course those occur infrequently: two midterms and the final. 


Table 3 


If you read the textbook what motivated you to do so? 

Traditional 

High Tech ! 

a. Better prepared for class 

11 

24% 

2 

7% 

b. Answer Quizzes 

15 

33% 

22 

79% 

c. Facilitates learning the material 

9 

20% 

1 

4% 

d. Figure out what the instructor said in class 

10 

22% 

3 

11% 


45 


28 



The real impact of the online quizzes can be seen in the timing of the students’ reading. Of the high tech 
students 93% (before and both) tended to read the material before class. The traditional students only 35% read the 
book before class. 


Table 4 


If you read the textbook did you read it before class or after? 

Traditional 

High Tech 1 

a. Before 

17 

35% 

25 

83% 

b. After 

31 

65% 

2 

7% 

Both 



3 

10% 


48 


30 



In terms of practicing the problems, the technology appears to have discouraged a larger percentage of 
students, 21% either got discouraged or did not use the study guide as compared to only 6 % in the traditional class. 
In the high tech class all the problems were online so if the students had trouble with using the web they became 
effectively foreclosed. 


Table 5 


I practiced the problems 

Traditional 

High Tech j 

a. all of the assigned problems 

8 

15% 

6 

18% 

b. most of the assigned problems 

21 

40% 

13 

39% 

c. some of the assigned problems 

21 

40% 

7 

21% 

d. tried to, but got discouraged 

1 

2% 

4 

12% 

e. did not use the study guide 

2 

4% 

3 

9% 


53 


33 



The traditional classroom model has an active learning exercise that uses the entire class period about once 
every two weeks. These exercises are generally fun (auctioning off money or conducting a prisoner’s dilemma style 
negotiation) however the lesson derived from the exercise may be less obvious than a more linear presentation of the 
material. 
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Table 6 


I found classroom activities (circle all that apply) 

Traditional 

High Tech j 

a. Enjoyable 

32 

63% 

10 

30% 

b. Interesting 

10 

20% 

15 

45% 

c. Instructional 

6 

12% 

16 

48% 

d. Moderately ineffective 

3 

6% 

5 

15% 

e. Very ineffective 

0 

0% 

0 

0% 


51 


33 



The greatest area of dissatisfaction is with the textbook. In the traditional course, 50% found the book 
ineffective or very ineffective, in the high tech course, 60% found the book ineffective or very ineffective. Part of 
this problem may be that some students chose not to purchase a hard copy of the book and therefore had to do their 
reading online which can be difficult. Note some of the students who found the book ineffective also found it 
instructional; 42% of the traditional students and 45% of the high-tech students found it instructional. In this area, 
both classroom approaches yield similar results. 


Table 7 


I found the textbook (circle all that apply) 

Traditional 

High Tech | 

a. Enjoyable 

0 

0% 

1 

3% 

b. Interesting 

3 

6% 

3 

9% 

c. Instructional 

20 

42% 

15 

45% 

d. Moderately ineffective 

20 

42% 

13 

39% 

e. Very ineffective 

5 

10% 

7 

21% 


48 


33 



How the students rank the lectures is difficult to analyze. Relative to auctioning off dollar bills, any normal 
classroom lecture will be duller. On the other hand, in the high tech course, if the online quizzes have compelled the 
students to read more thoroughly in advance, the lecture/discussion can focus on the more interesting applications 
and extensions of the lesson rather than conveying the nuts and bolts of the material. 


Table 8 


I found the lectures/discussion (circle all that apply) 

Traditional 

High Tech i 

a. Enjoyable 

12 

23% 

14 

42% 

b. Interesting 

12 

23% 

19 

58% 

c. Instructional 

24 

46% 

18 

55% 

d. Moderately ineffective 

3 

6% 

0 

0% 

e. Very ineffective 

1 

2% 

2 

6% 


52 


33 



The following table shows similar results for the two approaches: students don’t enjoy homework, they 
don’t find it interesting, they do it to learn the material and pass the exams. 
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Table 9 


I found homework (circle all that apply) 

Traditional 

High Tech j 

a. Enjoyable 

0 

0% 

0 

0% 

b. Interesting 

5 

10% 

4 

12% 

c. Instructional 

37 

73% 

19 

58% 

d. Moderately ineffective 

5 

10% 

11 

33% 

e. Very ineffective 

4 

8% 

4 

12% 


51 


33 



The high tech model resulted in students on average spending more time studying than in their other 
courses. In fact 30% spent much more time and 18% spent more time. That is nearly half the class spent more time 
with their economics course than with the other courses they were taking. This is probably attributable to the 
chapter quiz requirement which forces the students to read the book and answer the questions before class. In the 
traditional class 50% of the class spent less or much less time on their economics course. This compares with only 
12% of the high-tech students. Many students don’t read the book before class and after the lecture figure that they 
know it and just skim the material as evidenced by Table 2 above. 


Table 10 


How did this amount compare to the time you spent on your 
other courses 

Traditional 

High Tech 

a. Much more 

1 

2% 

10 

30% 

b. More 

3 

6% 

6 

18% 

c. About the same 

22 

42% 

13 

39% 

d. Less 

20 

38% 

3 

9% 

e. Much less 

6 

12% 

1 

3% 


52 


33 



The effort impact is clearly demonstrated in the table below which asks the students to estimate the average 
hours per week spent outside of class. The vast majority of traditional students, 63% of those responding, only 
spend an average of an hour a week on economics. In the high tech class only 13% are spending only an hour a 
week. The traditional class students average 1.8 hours per week while the high tech class students average 5.2 hours 
per week. In addition to the reading and quizzes, many of the high tech students spent time on the various practice 
problems and graphs developed specifically for the course which matched the exam questions they would answer on 
their in class exams. 


Table 11 


On average how many hours per week Outside of class did you 
spend reading and studying economics? 

Traditional 

High Tech 

1 Hour 

24 

4 

2 Hour 


5 

3 Hours 

8 

4 

4 Hours 

5 

4 

5 Hours 


3 

6 Hours 

1 


8 Hours 


1 

10 Hours 


5 

11 or more Hours (averaged as 12 hours) 


3 

Class average 

1.8 

5.2 
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Student assessment of their learning in the course showed large differences between the high tech class and 
the traditional class. Specifically in the high tech class 15% said that they learned much more than in their other 
courses compared to 4% in the traditional class. Additionally in the high tech class, 30% said that they learned more 
than in their other courses as compared to only 12% in the traditional class. 


Table 12 


In reflecting on what you learned how would you compare it to 
how much you learned in your other courses, (circle one 
answer) 

Traditional 

High Tech 

a. Much more 

2 

4% 

5 

15% 

b. More 

6 

12% 

10 

30% 

c. About the same 

30 

58% 

11 

33% 

d. Less 

12 

23% 

4 

12% 

e. Much less 

2 

4% 

3 

9% 


52 


33 



One of the problems with the high tech class is that some students are put off by the technology and don’t 
get help. This problem is evidenced by the responses below regarding the study guide. In the traditional class, 
students could just open the book. When the study guide is available online, the students have to access the web and 
then locate the correct material within the website. As we can see 27% of the students did not find the material on 
the web or got discouraged using it; only 6% did all the assigned chapters. In the traditional class, 55% did the study 
guide for all the assigned chapters. Aside from technological trepidation, a reason these number might be so 
different is that in the traditional class, the study guide is the assigned homework, whereas in the high tech class the 
reading and the quizzes were the assigned work; the study guide and other exercises were available options to use to 
study for exams. Students tend to do work that is specifically assigned and graded. 


Table 13 


Did you use any of the study guide? (circle one answer) 

Traditional 

High Tech j 

a. For all of the assigned chapters 

28 

55% 

2 

6% 

b. For most of the assigned chapters 

15 

29% 

6 

18% 

c. For some of the assigned chapters 

7 

14% 

16 

48% 

d. Tried to use them but got discouraged 

0 

0% 

4 

12% 

e. Did not find them on the web 

1 

2% 

5 

15% 


51 


33 



Table 14 


I found the study guide (circle all that apply) 

Traditional 

High Tech j 

a. Enjoyable 

4 

8% 

1 

3% 

b. Interesting 

7 

14% 

3 

9% 

c. Instructional 

38 

76% 

23 

70% 

d. Moderately ineffective 

1 

2% 

5 

15% 

e. Very ineffective 

4 

8% 

0 

0% 


50 


33 



Assessment of Digital Teaching Tools - Survey Results: 

Web-based assignments were created by Dr. Granitz, co-author of this paper; these assignments are 
introduced in the two courses to identify the importance of context, that is, the importance of the overall pedagogical 
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approach in the introduction of an innovative teaching tool. Digital graphs were created to depict various industry 
structures: perfect competition, monopolistic competition and monopoly in both the long run and short run. The high 
tech students had these digital graphs imbedded with the other online materials. The traditional students were 
emailed the set of digital graphs. While the majority of students in the traditional class had problems with using the 
digital graphs, many did try and did use them. Some students in the high tech class were also challenged by the 
digital format and only 12% used all of them. Of the students who used the online material throughout the semester 
78% did some, most, or all of the web exercises. Of the traditional class, 47% used them to some extent. 


Table 15 


I did the digital exercises (the graphs I e-mailed you) 

Traditional 

High Tech 

a. all of the web exercises 

6 

12% 

3 

9% 

b. most of the web exercises 

5 

10% 

14 

42% 

c. some of the web exercises 

13 

25% 

9 

27% 

d. tried but got discouraged 

8 

16% 

4 

12% 

e. did not use the web exercises 

19 

37% 

3 

9% 


51 


33 



Almost half of the high tech students, who had digital material all semester, felt that the digital graphs 
helped them participate more in class and perform better on tests whereas about a third of the traditional students 
had that response. 


Table 16 



Traditional 

High Tech j 


Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Do you believe the digital material helped you to 
participate more in class? 

17 

34 

33% 

67% 

15 

18 

45% 

55% 

Do you believe the digital material helped you 
perform better on tests? 

18 

33 

35% 

65% 

16 

17 

48% 

52% 


In the high tech class the vast majority of the students (70%) found the digital graphs instructional. 21% of 
the high tech students did not use them or got discouraged (Table 15, above); this means that of the students who 
used them, 85% found them instructional, and some of those also found them interesting and enjoyable. While half 
of the traditional students found the web exercises instructional, a larger proportion found them ineffectual. This is 
to be expected. When one introduces a pedagogy that is less familiar, it is less well received, regardless of the value 
of the pedagogy. Just as collaborative work is better received when done consistently throughout the semester, high 
tech exercises will be more effective if integrated throughout the course. It should be that 66% of the traditional 
students who used the graphs found them instructional, 24 traditional students used the digital graphs (Table 15 
above) 16 found them instructional (Table 17. below). 


Table 17 


I found the digital graphs (circle all that apply) 

Traditional 

High Tech j 

a. Enjoyable 

2 

6% 

4 

12% 

b. Interesting 

7 

22% 

6 

18% 

c. Instructional 

16 

50% 

23 

70% 

d. Moderately ineffective 

9 

28% 

2 

6% 

e. very ineffective 

10 

31% 

2 

6% 


32 


33 
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In both courses many of the students who did use the digital graphs found them helpful or very helpful. In 
the high tech class only 10% found them unhelpful or a waste of time compared to 38% in the traditional class, 
again in analyzing these results we should remember that only 24 students said they used the graphs. 


Table 18 


I found the digital graphs (circle all that apply) 

Traditional 

High Tech | 

a. Very helpful 

1 

2% 

10 

34% 

b. Helpful 

17 

35% 

7 

24% 

c. Neutral 

12 

25% 

9 

31% 

d. Unhelpful 

12 

25% 

2 

7% 

e. A waste of time 

6 

13% 

1 

3% 


48 


29 



Assessment of Survey Results - Conditioned on Student Effort 

We take into consideration the amount of effort expended by the student. First we look at the answers of 
those who actually read the book. Of the high tech students, 50% think it helped them on tests and 30% thought it 
enhanced classroom participation, this differs from the entire high tech class perception where 26% thought it would 
help with tests and 44% thought it would help classroom participation. So apparently those who didn’t read the book 
thought it wouldn’t help much and more of those who did read the book found it useful. In the traditional setting, of 
those who read the book, 30% did so to help prepare for the test while over 50% did so to prepare for class. Again 
these numbers are slightly higher that the entire class, so those who read the book confirm that it is useful. 

The differences shown below between the two classroom environments show that the high tech students are 
motivated by the desire to do well on the tests. 


Figure 1 


Conditional - Given Reading the Text 



Hi Technology 
II Traditional 


Next we assess student effort by the amount of time the students devote to this course relative to other 
courses; they may spend more time on this course, relative to others, the same amount, or less time than others. If 
they spend more time, do they perceive that they get more out of it? 
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Figure 3 

Time Spent with Instructional Class 



Looking at the students that spend more time in their high tech course than in others, about 12 % found it 
enjoyable, just over 20 % found it instructional and fewer than 10% found it ineffectual. There were fewer students 
in the traditional course that spent more time in this course, resulting in numbers less than 5% in each case. In both 
sets of classes, about 40% of the students spent the same amount of time preparing for microeconomics, the 
Uaditional students enjoyed it more and the high tech found it more instructional. Of the students that spent less time 
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on this course, there were very few high tech students and the percentages hover around 5% in each case. The 
traditional students who spent little time were able to enjoy class (25%) and found it instructional (20%). This last 
difference highlights the fact that the high tech students had to prepare whereas the traditional students were able to 
get something out of class despite inadequate preparation. 

Conclusions From The Survey: 

The survey shows that using a variety of online tools and online chapter quizzes can increase the amount of 
time devoted to a course, and the perceived learning in introductory economics. The caveat is that one can not 
simply introduce high tech exercises ad hoc. They are most effective when couched in an ongoing high tech 
learning environment. This effectiveness will of course be compromised if the students are uncomfortable with the 
high tech learning environment. The drop rate for the high tech course this year is 25%; this may be due to the 
technology or perceptions that the instructor is more demanding than others. The traditional class had zero drops. 
This may be due to the fact that the course activities are engaging or because the instructor teaches most of the 
microeconomics classes and there are few alternatives! 

CONCLUSIONS 

From the instructor’s point of view, once the quizzes are set up and any additional material is developed, 
most of it can be reused for several years as these are basic microeconomic concepts. The graded online quiz 
encourages the students to engage the material prior to class and does not take any time away from lecture nor does 
it require any time to grade. The grades are automatically recorded in the student grade book. So the cost of set up is 
there, but it can be spread over many semesters and the ongoing costs are minimal. This entire process has gotten 
much easier for instructors over the past few years as more and more publishers are creating good online tools to 
complement either a hardcopy book or an online book. The benefit is that the students come to class with a better 
idea of the basic concepts in each chapter and so more time can be spent evaluating the models and discussing the 
insights these models provide about the world. This provides the opportunity to have more of a discussion/dialogue 
or a collaborative learning format rather than straight lecture. The graded quizzes provide immediate feedback to 
both the student and instructor; the instructor can review the quiz results and determine which concepts a large 
number of students did not understand. Another advantage of the online quiz is that it forces the students to go 
online and once there they may try some of the other online tools. In courses where there is no compelling reason to 
go online, students tend to ignore the supplemental material there. 

Innovative pedagogy, whether through active learning exercises or online material is needed to reach a 
more diversified student body, accommodating various learning styles. It is more is stimulating and a stimulating 
environment is conducive to learning. We are making efforts to extend the student’s learning experience and this is 
our initial attempt to assess the impact of the various tools in two different environments. 
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Taskforce. Granitz developed a number of successful economics learning objects and integrated them with 
commercially available web-based tools to enhance the educational experience of her students. 
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